{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **演示0404：反向传播图解法**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在链式求导过程中，如果“链条”较长，那么很容易出现错漏。图解法提供了正向计算和反向求导计算的流程图，有助于理解求导过程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **案例1：**\n",
    "> \n",
    "已知$f(x, y, z)=(x+y) \\cdot z$，请计算$x=-2, y=5, z=-4$时的导数$\\dfrac{\\partial f}{\\partial x}$，$\\dfrac{\\partial f}{\\partial y}$和$\\dfrac{\\partial f}{\\partial z}$\n",
    "* 直接计算  \n",
    "令$q=(x+y), f=qz$，很显然有：  \n",
    "$ \\dfrac{\\partial f}{\\partial x}=\\dfrac{\\partial f}{\\partial q} \\frac{\\partial q}{\\partial x}=z \\cdot 1=z=-4 $  \n",
    "$ \\dfrac{\\partial f}{\\partial y}=\\dfrac{\\partial f}{\\partial q} \\frac{\\partial q}{\\partial y}=z \\cdot 1=z=-4 $  \n",
    "$ \\dfrac{\\partial f}{\\partial z}=q=x+y=3 $\n",
    "* 画出导数计算流图如下：  \n",
    "![title](../images/040401.png)  \n",
    "* 解释\n",
    " * 上图中，横线上方的绿色数值代表该变量的值，横线下方的红色数值代表该标量相对最终计算结果$(f)$的导数值\n",
    " * 最右边节点相对自己的导数恒为1\n",
    " * 从右向左计算各个节点的导数\n",
    " * 两个变量相乘，其针对最终计算结果$(f)$的导数，等于下游(右侧)节点导数值乘以另一个变量。（有置换的效果）\n",
    " * 两个变量相加，其针对最终计算结果$(f)$的导数，等于下游节点导数值。相当于下游节点的导数完整的流回给各个变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **案例2：**\n",
    ">  \n",
    "已知$f(w,x)= \\dfrac{1}{1+e^{-(w_0 x_0+w_1 x_1+w_2)}}$，使用导数计算流计算当$w_0=2.0, x_0=-1.0, w_1=-3.0, x_1=-2.0, w_2=-3.0$时的各个导数值  \n",
    "* 回顾下列常用导函数  \n",
    "$ f(x)=\\dfrac{1}{x} => f'(x)=-\\dfrac{1}{x^2} $  \n",
    "$ f(x)=e^x => f'(x)=e^x $\n",
    "* 做出导数流图  \n",
    "![](../images/040402.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **案例3：**\n",
    "已知$W$为$(N,M)$矩阵，$x$为$M$维列向量，$f(W,x)=||Wx||^2=\\sum_{i=1}^{N}{(Wx)_i}^2 $，计算当$W=\\left(\\begin{array}{c}0.1&0.5\\\\-0.3&0.8\\end{array}\\right),x=\\left(\\begin{array}{c} 0.2 \\\\ 0.4\\end{array}\\right)$时的$\\dfrac{\\partial f}{\\partial W}$和$\\dfrac{\\partial f}{\\partial x}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">**推导导数计算公式：**  \n",
    "* 设置中间变量$y$  \n",
    "令：$ y=Wx=\\left(\\begin{matrix}\n",
    "w_{11} x_1+w_{12} x_2+ \\cdots +w_{1m} x_m \\\\\n",
    "w_{21} x_1+w_{22} x_2+ \\cdots +w_{2m} x_m \\\\\n",
    "\\vdots \\\\\n",
    "w_{n1} x_1+w_{n2} x_2+ \\cdots +w_{nm} x_m\n",
    "\\end{matrix}\\right)\n",
    "=\\left(\\begin{matrix}\n",
    "y_1 \\\\ y_2 \\\\ \\vdots \\\\ y_n\n",
    "\\end{matrix}\\right) $  \n",
    "显然，$y$是$N$维列向量  \n",
    "* 计算$\\dfrac{\\partial f}{\\partial y}$  \n",
    "$ \\begin{aligned}\n",
    "f= & (w_{11} x_1+w_{12} x_2+\\cdots+w_{1m} x_m )^2+(w_{21} x_1+w_{22} x_2+\\cdots+w_{2m} x_m )^2+\\cdots+ \\\\ & (w_{n1} x_1+w_{n2} x_2+\\cdots+w_{nm} x_m )^2 \\\\ =& \\sum_{k=1}^{n}{y_k}^2 \n",
    "\\end{aligned}$  \n",
    "可得：$ \\dfrac{\\partial f}{\\partial y} = 2y $  \n",
    "* 计算$\\dfrac{\\partial y}{\\partial W}$和$\\dfrac{\\partial y}{\\partial x}$  \n",
    "回顾上一实验中的矩阵链式求导公式：若$y=Wx$，则：  \n",
    "$\\dfrac{\\partial y}{\\partial W}=x^T$  \n",
    "$\\dfrac{\\partial y}{\\partial x}=W^T$  \n",
    "* 最终结果：\n",
    " * $\\dfrac{\\partial f}{\\partial W} = \\dfrac{\\partial f}{\\partial y} * \\dfrac{\\partial y}{\\partial W} = x^T * 2y$\n",
    " * $\\dfrac{\\partial f}{\\partial x} = \\dfrac{\\partial f}{\\partial y} * \\dfrac{\\partial y}{\\partial x} = W^T * 2y$  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">**作出计算图：**  \n",
    "* 正向计算：  \n",
    "![](../images/040403.png)    \n",
    "* 反向求导：  \n",
    "![](../images/040404.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
